Citations of microRNA Biomarker Articles That Were Retracted

This systematic review investigates the association of retractions with patterns of further citations among articles reporting microRNA studies.


Introduction
Retraction is a means that academic publishers can use to deter research misconduct and alert audience of erroneous content published in the journals. 1Retraction Watch recorded that the number of retracted articles increased from 41 articles in 2000 to more than 6000 articles in 2022. 2 Nature reported that the number of retractions increased each year and passed 10 000 articles in 2023. 3The increasing prevalence of documented retractions outpaced the growth rate of publications. 3The situation may be more severe given that that some researchers believe that documented retractions are just the tip of the iceberg. 4Common reasons given for retractions included honest scientific mistakes (eg, inadvertent errors), 1 research misconduct (eg, plagiarism, fabrication, or falsification), 5 and various administrative errors. 69][10][11] The International Committee of Medical Journal Editors (ICMJE) recommends PubMed as the verified bibliographic source for references to minimize citation errors and an authoritative source of information about retractions for articles indexed in MEDLINE. 12A PubMed search found that more than 6000 articles were retracted between 2020 and 2022 and that cancer, RNA, and microRNA were the most frequently appearing words in retractions after excluding words like retraction (or retracted), note, cell, pathway, human, patient, inhibits, targeting, and promotes.In the Retraction Watch database, 232 of 484 retractions (47.93%) were about microRNAs in genetics in 2020, and this proportion increased to 699 of 1026 retractions (68.13%) in 2022. 2 These outcomes suggest that microRNA research may be problematic as a source of retractions.
Articles in 2020 13 and 2021 14 found that retracted articles were often cited as if they had never been retracted, although the ICMJE recommended that authors are responsible for checking that none of their references cite retracted articles except in the context of referring to the retraction. 12t many journals check references in manuscripts for postretraction citations.Only recently, Wiley implemented a reference checking system to access the Retraction Watch database. 15 our knowledge, few articles have covered postretraction citations to evaluate the effectiveness of retractions in reductions of further citations of retracted articles.This study aimed to fill this research gap by conducting a systematic review to identify characteristics of retractions in microRNA biomarker research as a common source of recent retractions.Specifically, we examined trends of postretraction citations by comparing retracted microRNA biomarker articles with a control group of nonretracted articles and investigated whether retraction was associated with reductions in postretraction citations.

Methods
This systematic review was registered in the Open Science Framework (ME89S) in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.The Third People's Hospital of Chengdu determined that this study was exempt from ethics approval and consent to participate because personal information was not involved.

Design and Search Strategies
Web of Science, PubMed, and Retraction Watch Database were searched from their inception to July 17, 2021, to identify retracted publications on microRNA.Retraction, microRNA, withdrawal, and their synonyms were used as search terms.Detailed search strategies for each database are shown in the eMethods in Supplement 1.Additionally, PubPeer was referenced to examine the public response or comments on included retractions.

Eligibility Criteria and Study Selection
Studies meeting the following criteria were included: they were about microRNA research; they were retracted articles regardless of reasons; and they were journal articles not conference abstracts.No restriction was imposed on language, participants, intervention, or comparison.Eligible articles were screened and selected by 2 investigators (H.Z. and Y.J.) according to these inclusion criteria independently.Disagreements between investigators were discussed with the third investigator (S-W.L.) until an agreement was reached and recorded with rationales.

Data Extraction and Selection as Control Studies
The following characteristics of eligible articles were extracted: title, journal, publisher, specific microRNA, disease, affiliations, reasons for retraction, number of authors, dates of publication and retraction, nature of retraction notice, and country of authors.Retraction reasons were defined and categorized according to Retraction Watch Database User Guide Appendix B: Reasons. 16Additionally, Web of Science was used to retrieve data on citations of retracted articles, along with the journal impact factor (JIF) of the retracting journal for the calendar year before publication and for 2020 and its 2020 five-year JIF.To investigate the association of the retraction with further citations, a random selection of 10% of retracted articles (group A) based on an arbitrary seed number (set to 5 in this study) was performed (articles were ranked according to their publication date), and an analysis of their detailed citations was conducted.A set of nonretracted control articles (group B) from the same journals in the same year and month of publication and with the same number of authors was identified in PubMed.Studies citing retracted articles in group A were labeled as group C. The same method used to identify group B articles was used to find another set of control articles (group D) for group C (articles citing retractions); however, group D included articles regardless of whether they were retracted.The study design is shown in Figure 1.

Statistical Analysis
Statistical analysis was performed with R statistical software version 4.2.2 (R Project for Statistical Computing). 17Continuous data following a normal distribution were presented as means with 95% CIs and compared using the Student t test.Continuous data that do not follow a normal distribution are expressed as median (IQR) and compared using the Mann-Whitney U test.Categorical data were reported as counts and percentages and compared using a χ 2 test.All statistical tests were 2-sided, and values of P < .05were considered statistically significant.Quality assessment was not conducted on retracted articles owing to the varieties of misconduct and removal of the full text by journals.
Data were analyzed from September 2021 through March 2023.

Retracted microRNA Studies and Their Characteristics
A total of 4427 records were identified from PubMed, Web of Science, and Retraction Watch Database.After removal of duplicates, 1303 records were available for screening of abstracts.The screening excluded 20 records that were not about microRNA and 274 records that were not retracted or withdrawn articles.Finally, 887 records met eligible criteria after we excluded 10 records of conference abstracts and 112 records of notices for correction or expression of concern (Figure 1).
Of included records, 766 articles (86.36%) and 740 articles (83.43%) were labeled as retractions by Retraction Watch and PubPeer, respectively.There were 435 articles that received comments or attention in PubPeer (49.04%).Retracted articles were published between 1999 and 2021, with 36 articles not providing a retraction year.The first retraction of microRNA articles was in 2003 and initiated by Nature. 18As shown in eFigure 1 in Supplement 1, the number of retracted articles increased by year and articles published in 2019 had the most retractions.In accordance with preestablished criteria (eMethods in Supplement 1), 9574 control articles were identified, for a total of 10 461 articles. The

Retraction Characteristics
Among retracted articles, the exact date of retraction was unknown for 98 articles (11.05%).For 789 articles with reported retraction dates, the median (IQR) time from publication to retraction was 22.03 (10.30-39.00)months.There were 847 articles retracted directly, while in 40 articles (4.51%), the journal published a correction or expression of concern before retraction.The time from notice of correction or concern to retraction varied, with a median (IQR) time of 8.9 (3.975-10.95) months.
The shortest time from correction or expression of concern to retraction was less than 1 month, while the longest time was more than 6 years.Analysis of retraction reasons found that 870 articles (98.08%) were retracted because of 1 to 6 reasons, with a small proportion of articles retracted for more than 6 reasons.Data problems (including data in the form of images) and publication by paper mills were the main reasons that articles were retracted ( We investigated the number of retractions and JIF and found that retractions tended to occur in journals with a low or no impact factor (Figure 2).

Citations
Contrary to the expectation that retractions would be associated with reduced citations, retracted articles included in this systematic review were cited 6327 times after retraction (Table ).Among 792 retracted articles with citations, 621 articles (78.41%) were cited at least once after retraction.
Approximately 30% of retracted articles with citations (238 articles [30.05%]) were cited more after retraction than before retraction, including in top journals as ranked by Journal Citation Reports (eg, Nature), 19 4).Among all citations, 332 citations (69.46%) were in research articles, 141 citations (29.50%) were in reviews, and 2 citations (0.42%) were in proceedings of meetings; there was 1 citation (0.21%) each in a letter, editorial, and retraction notice.Among reviews, most citations were in the main text (136 citations [96.45%]), and among research articles, most citations were in the introduction (153 citations [46.08%]) and discussion (192 citations [57.83%]).However, 14 research articles (2.93%) cited the retraction in the methods section on data analysis or to establish models; 1 of these articles also incorporated the data in a new analysis.To investigate the association of retraction with citations, a total of 1620 nonretracted articles (group B) were identified in PubMed as controls for the 87 retracted articles (group A).No significant difference was found between retracted articles and their nonretracted control articles in overall citations or citations after retraction (eTable 2 in Supplement 1), but there were significant differences in citation difference (citations after retraction minus citations before retraction; mean  The x-axis represents different time windows between retraction and citation retrieval (September 2021).The left y-axis shows the number of articles in each time window excluding those with zero citations between retraction and citation retrieval.Overall citations comprise citations before and after retraction.that medical doctors were driven by substantial incentives, such as job promotion and cash rewards.Given that Chinese authorities stopped paying researchers cash rewards in 2020 21 and stopped promoting or recruiting staff solely on the quantity of their publications, it seemed that Chinese authorities had been aware of the integrity issues of medical research in hospitals.In addition, the National Health Commission of China (in charge of hospitals) set up a special committee on June 3, 2021, to strengthen the integrity of medical research in hospitals. 22However, their new measures did not address the issue of postretraction citations.
Although some publishers have retracted a large number of articles from paper mills within a short time after publication, approximately 70% of included retracted microRNA articles were still cited by new articles after their retraction.Furthermore, approximately 30% of retracted articles (238 articles [30.05%]) were cited more frequently after retraction than before retraction.
Unexpectedly, 43.51% of citations happened 12 months after retraction and 19 of 478 citing articles (3.97%) noted the retraction.The high rate of citation of articles after retraction was also found in a previous report 23 and similar articles in other fields, 13,14 which suggest that postretraction citation may be common and that not enough attention has been paid by some authors to retracted articles when citing articles.Overall citations (comprising citations before and after retraction) and postretraction citations accumulated over time.Furthermore, there were no significant differences between the sample of retracted articles and the control sample of nonretracted articles, regardless of overall citations and citations after retraction.In other words, retraction was not associated with changes in further citations, which echoes results of the study by Peng etal 24 that retraction was not associated with a reduction in negative outcomes in science associated with problematic articles.
The reason why retraction was not effective may be partly explained by the study by Xu et al, 25 which compared official information channels (ie, releases of information through authorities, such as government agencies or organizations that have power in a particular area) with unofficial information channels (ie, contents posted are not released by the authorities) in disseminating retraction information from January 2005 to December 2014, finding that official channels were ineffective.Xu et al 25 found that unofficial information channels provided additional retractionrelated information (ie, retraction reasons and details of the retraction investigation processes) compared with official channels, which would be associated with reduced postretraction citations. 25e Committee on Publication Ethics (COPE) published the COPE Retraction Guidelines in November 2019, suggesting that official notices of retraction should state reasons for retraction. 1 However, the inadequacy of official notices for retraction (eg, a lack of retraction reasons) was also found in biomedical literature retracted after 2019 (ie, in 2020 and 2021) in our study.Additionally, the

Figure 3 .
Figure 3. Mean and Median Citations by Time Window After Retraction 200

Figure 4 .
Figure 4. Percentages of Retraction-Unaware and Retraction-Aware Citations After Retractions 50 Table shows characteristics of retracted articles on microRNAs.Among all retracted

JAMA Network Open | Medical Journals and Publishing for
Table; eTable 4 and eFigure 2 in Supplement 1).Retractions of microRNA articles were made by 62 publishers and 204 journals.As shown in eTable 1 in Supplement 1, 16 publishers (25.81%) contributed 777 retractions (87.60%).One reason this proportion was that the 16 publishers had more journals involved than the other 46 publishers.More importantly, the top 17 journals with the most retractions were published by these 16 publishers.As shown in eTable 1 in Supplement 1, the European Review for Medical and Table.Characteristics of Retracted Studies Pharmacological Sciences alone contributed 122 retractions, which was more than the 110 retractions by 46 publishers with 69 journals; Journal of Cellular Biochemistry was second, with 91 retractions.
Abbreviation: JIF, journal impact factor.a Citation was marked as not available if the study was not indexed by Web of Science.b Organizations selling fake work and authorships to researchers.JAMA Network Open | Medical Journals and Publishing Citations of microRNA Biomarker Studies That Were Retracted JAMA Network Open.2024;7(3):e243173.doi:10.1001/jamanetworkopen.2024.3173(Reprinted) March 21, 2024 5/12 Downloaded from jamanetwork.comby guest on 03/23/2024 general Science Citation Index (SCI) and SCI-Expanded (SCIE) journals (eg, Neuroscience Letters), and journals that were not indexed by SCI or SCIE (eg, Genetics and Molecular Research).As shown in the Table, approximately 70% of retracted articles with citations had at least 1 citation after retraction (621 articles [70.01%]) and 6 articles had more than 100 citations after retraction.We examined the mean and median citations in different time windows after retraction and found that overall citations (comprising citations before and after retraction) and postretraction citations accumulated over time (eg, the median [IQR] number of postretraction citations was 1 [1-2] and 23[9-44]citations at the first 6 and 66 months, respectively, between retraction and citation retrieval) (Figure3), suggesting that retraction was not associated with reductions in citations of retracted articles.Citations after retraction were further analyzed in a random selection of retracted articles (89 articles [10.03%]) to investigate whether citing authors noted the status of publications (ie, the retracted state).There were 478 citations after the retraction of 87 articles (9.81%; excluding 2 articles without a date of retraction); 208 citations (43.51%) happened 12 months after retraction, and 19 citing articles (3.97%) noted retractions (Figure P = .001)(eTable 2 in Supplement 1) and odds of being cited more after retraction than before retraction (odds ratio, 0.62; 95% CI, 0.40-0.96)(eTable 3 in Supplement 1).To test if articles citing retractions were more likely to be themselves retracted later, a set of 7954 articles (group D) that did not cite the 87 retracted articles were compared with 414 articles (group C) among 478 citing articles (7 citing articles were not found in PubMed, and no control articles were identified for 57 of 478 articles citing retractions).Of 414 articles citing retractions, 12